Papers with image representation
Vector of Locally-Aggregated Word Embeddings (VLAWE): A Novel Document-level Representation (N19-1)
Copied to clipboard
| Challenge: | a novel word embedding representation for text documents is proposed . the method is based on the Vector of Locally-Aggregated Descriptors used for image representation . |
| Approach: | They propose a novel representation for text documents based on aggregating word embedding vectors into document embeddables. |
| Outcome: | The proposed representation improves on the Movie Review data set and is 10% better than the state-of-the-art representation. |
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language pre-training methods use a two-step training procedure to learn visual features from image-text pairs. |
| Approach: | They propose a vision-language pre-trained model for V+L understanding and generation using a unified Transformer framework. |
| Outcome: | The proposed model can learn visual representation and semantic alignments between image and text on visual-text pairs and on visual processing tasks. |
Object Counts! Bringing Explicit Detections Back into Image Captioning (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to image captioning use explicit object detectors as an intermediate step, but they bypass the explicit detection phase and instead generate captions directly from image embeddings. |
| Approach: | They argue that explicit detections provide rich semantic information and can thus be used as an interpretable representation to better understand why end-to-end image captioning systems work well. |
| Outcome: | The proposed methods can be used to understand why end-to-end captioning systems work well. |
Improving Image Captioning with Better Use of Caption (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to image captioning focus on visual attention, but many do not. |
| Approach: | They propose a framework that explores semantics available in captions and leverages that to enhance both image representation and caption generation. |
| Outcome: | The proposed framework outperforms baselines on the MSCOCO dataset and is state-of-the-art under a wide range of evaluation metrics. |
Can VLMs Recall Factual Associations From Visual References? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic deficiency in the multimodal grounding of Vision Language Models is identified . VLMs can recall factual associations when provided a textual reference to an entity . |
| Approach: | They identify a systematic deficiency in the multimodal grounding of Vision Language Models . they show that VLMs struggle to link their internal knowledge of an entity with its image representation . |
| Outcome: | The study shows that VLMs struggle to link internal knowledge with image representations . the findings provide recommendations for future research . |